Image processing system based on stereo image

ABSTRACT

The present invention relates to an image processing system for pointing or making an augmented reality image by selectively capturing a left or right image of a stereo image of a 3D display, and detecting a mark from the captured image.

TECHNICAL FIELD

The present invention relates to image processing system for pointing ormaking augmented reality image by selectively capturing left or rightimage of stereo image of 3D display and detecting mark from the capturedimage.

BACKGROUND ART

The present invention relates to image processing system for pointing ormaking augmented reality image using 3D display.

DETAILED DESCRIPTION OF THE INVENTION Technical Problem

There is the Korean patent: its registration number is 10-0936816 andtitle of the invention is ‘POINTING DEVICE USING CAMERA AND OUTPUTTINGMARK’ which is about a pointing device by capturing and detecting mousecursor icon displayed on computer monitor. The mouse cursor or markdisplayed on monitor can be detected by image processing means forpointing task. It has the weakness that detecting mark is difficult ifthe color of mark and background is similar.

There is the invention: its Korean patent application number is10-2011-0050980 and the title is ‘MONITOR BASED AUGMENTED REALITYSYSTEM’. this invention is relates to capturing image of mouse cursoricon or mark displayed on monitor, detecting mark and calculating the3-dimensional relative position between camera and mark by imageprocessing means for synthesizing the augmented reality image. Thisinvention also has the same weakness if the color of mark and thebackground is similar.

Technical Solution

The present invention provides the image processing system for fast andstably detecting mark from the captured image by displaying backgroundimage and mark image as stereo image with 3D display, capturing the markimage and detecting the mark.

Advantageous Effects

By using the image processing system of present invention, it ispossible to detect mark fast and stably with background image whosecolor is similar to the color of mark.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an image processing system according to an embodimentof the inventive concept.

BEST MODE

There are various types of 3D displays: shutter glass, polarizationglass, active retarder, red-cyan glass. The details are as followings:

Embodiment 1

3d display of shutter glasses type displays image for left eye and imagefor right eye sequentially. Shutter glasses opens and closes the shutterwith synchronization signal from 3D display. Wearer of the shutterglasses can watch left image with left eye and can watch right imagewith right eye. Such shutter glasses 3D display is commercialized bySAMSUNG. The 3D display can be LCD or beam projector.

In present embodiment, background image and mark image are sequentiallydisplayed as left and right image of stereo image and camera capturesonly the mark image with shutter synchronized with the 3D display. Themark is detected from the captured image by image processing means andthe augmented reality image can be synthesized with the 3-dimensionalrelative position between camera and mark.

The image processing means can control the outputting position of markby moving it to the line of sight of 1st camera so that the mark ispositioned at the center of captured image. By doing this, mark can bestably detected even if the 1st camera is moving. And the imageprocessing means can detect the size of mark from the captured image andcontrol the outputting size of mark so that the size of mark isproportional to the distance between camera and mark. The size of markin captured image is constant. In other words, it is recommended toreduce the displayed mark size if the 1st camera is approached to themonitor and to expand the displayed mark size if the 1st camera isgetting away from the monitor.

Instead of displaying left image of background and right image of marksequentially, it is also possible to display left image of 1stbackground, right image of 2nd background and the 3rd image of marksequentially and capturing the only 3rd image to detect mark from thecaptured image and use the detected information for pointing oraugmented reality. Wearer of shutter glasses that passes the only theleft and right image and blocks the 3rd mark image can watch the onlybackground image and cannot watch the mark. In this case the left andright image can be ordinary 3D stereo image.

MODE OF THE INVENTION Embodiment 2

Polarization glasses type 3D display is a display composed of thinhorizontal lines. The odd lines and even lines have differentpolarization coating. Left image and right image of stereo image aredisplayed at the even and odd lines respectively. Even and odd lineshave different phase retarder coating so that the beam passing throughthe even and odd lines can be polarized as different circularpolarization. Such a display is called patterned retarder type. Wearerof the polarization glasses can watch the 3D stereo image displayed bythe above polarization 3D display. In other words, the left lens ofglasses passes only the beam from even lines and right lens of glassespasses only the beam from the odd lines. Such a display can be LCD orbeam projector. For beam projector, left image beam and right image beamis projected at the same time and onto the same screen where the leftand right beam have the different polarization. It is recommended to usesilver screen to reflect the polarized beam.

By displaying the background image and mark image as stereo image withthe polarization type 3D display, capturing the only such mark imagewith polarization filter in front of camera, and detecting the mark fromthe captured image and calculating the 3D relative position between markand camera, such 3D relative position can be used for pointing task orsynthesizing the augmented reality image.

Embodiment 3

There are active retarder type 3D display which is the combination ofshutter glasses type and the polarization glasses type. The activeretarder 3D display system uses the polarization glasses instead ofshutter glasses to remove the battery for shutter glasses. It canovercome the low resolution problem of patterned retarder polarizationtype 3D display. The active retarder system utilizes the electricallycontrollable phase retarder installed in front of display which displaysthe left and right image of stereo image sequentially. Left and rightimage can be polarized differently by controlling the phase retarderwith synchronization signal from monitor. Wearer of polarization glassescan watch the 3D stereo image with the active retarder type 3D display.

Such active retarder display can be LCD or beam projector. For beamprojector, left and right image beams are projected sequentially withdifferent polarization. It is recommended to use silver screen toreflect the polarized beam.

By displaying the background image and mark image as stereo image withthe active retarder type 3D display, capturing the only such mark imagewith polarization filter in front of camera, detecting the mark from thecaptured image and calculating the 3D relative position between mark andcamera, such 3D relative position can be used for pointing task orsynthesizing the augmented reality image.

Embodiment 4

Wearer of red-cyan glass can watch 3D stereo image with monitordisplaying color image whose red component is left image of stereo imageand cyan component is right image of stereo image.

By displaying color image whose red component is background image andcyan component is mark image, capturing the displayed mark image withcyan color filter in front of camera, detecting the mark from thecaptured image and calculating the 3D relative position between mark andcamera, such 3D relative position can be used for pointing task orsynthesizing the augmented reality image.

There is dolby glasses type 3 display which is similar to the red-cyanglass type. Left and right lens of dolby glasses have slightly differentset of red, green, blue filter. By outputting left image in 1st set ofcolor and right image in 2nd set of color, wearer of dolby glasses canwatch the stereo 3D image in full color.

Embodiment 5

It is possible to construct augmented reality system by adding 2ndcamera into the composition of embodiment 1˜4 as show in FIG. 1. In suchaugmented reality system, 1st camera of embodiment 1˜4 captures only themark image with filter or shutter and image processing means calculatesthe 3 dimensional relative position between camera and mark. Theadditional 2nd camera captures only the background image with filter orshutter and the image processing means synthesizes the augmented realityimage by adding graphic object onto the captured background image withthe calculated 3 dimensional relative position information. Thesynthesized augmented reality image is more realistic because it doesnot contain the mark. It is recommended to fix the relative positionbetween 2 cameras. 2 cameras can be contained in smart phone or headmount display. The head mount display with camera is commercialized asWRAP 920AR+ (http://www.vuzix.com/consumer/products_wrap920ar.html). Onecamera of such head mount display can capture only the mark and anothercamera can capture the background image without mark and the synthesizedaugmented image can be displayed by the head mount display.

Embodiment 6

Camera of embodiment 1˜4 can be a gun shape for pointing task and theglasses can block the mark image and pass the background image withpolarization filter or shutter lens. For example, by displaying shootinggame scene as background image for left eye and mouse cursor icon asmark image for right eye on 3D display, capturing only mark image by gunshaped camera with filter or shutter, and detecting mark from thecaptured image by image processing means, user can play shooting game.It is recommended to wear shutter of polarization glasses which blocksthe mark image but pass background image.

Embodiment 7

Video-through head mount display of embodiment 5 can be replaced withthe transparent See-Thru Augmented Reality Display. In this case, the2nd camera of embodiment 5 can be omitted. Such a transparent head mountdisplay is commercialized as STAR 1200(http://www.vuzix.com/ar/products_star1200.html). By capturing the markimage with the camera attached to such a transparent head mount display,and detecting the mark from the captured image by image processingmeans, graphic object of augmented reality can be displayed on thetransparent display. Filter or shutter can be attached to thetransparent display in order to block the mark image and pass thebackground image. By doing so, user can watch the graphic object ofaugmented reality with background image which does not contain markimage.

1. An image processing system comprising: a display portion fordisplaying a mark in a specific image of a stereo image; a cameraportion including a first camera portion for capturing the specificimage of the stereo image; and an image processing portion for detectingthe mark from the specific image captured by the first camera portion.2. The image processing system of claim 1, wherein the image processingportion detects the mark and outputs a pointing signal or synthesizes agraphic object for an optical see through augmented reality system. 3.The image processing system of claim 1, wherein the camera portionincludes a second camera portion for capturing an image which does notcontain the mark, and the image processing portion synthesizes anaugmented reality image onto the image captured by the second cameraportion using the mark from the image captured by the first cameraportion.
 4. The image processing system according to claim 2, whereinthe image processing portion moves position of the mark toward a gazedirection of the first camera portion so that the position of the markin the specific image captured by the first camera portion remains atthe center of the specific image.
 5. The image processing systemaccording to claim 2, wherein the image processing portion adjusts sizeof the mark in proportion to distance between the camera portion and themark so that the size of the mark in the specific image captured by thefirst camera portion remains at a constant value.
 6. The imageprocessing system of claim 1, wherein the display portion outputs a markimage and a background image without a mark, sequentially, and the firstcamera portion captures only the mark image by using shutter means. 7.The image processing system of claim 1, wherein the display portionoutputs a mark image and a background image without the mark,polarization of the mark image and polarization of the background imageare different, and the first camera portion captures only the mark imageby using a polarization filter.
 8. The image processing system of claim1, wherein the display portion outputs a mark image and a backgroundimage without the mark, color of the mark image and color of thebackground image are different, and the first camera portion capturesonly the mark image by using a color filter.
 9. The image processingsystem of claim 1, wherein the display portion outputs a mark image anda background image without the mark, color of the mark image and colorof background image are different, and the image processing portiondetects the mark from the specific image captured by the first cameraportion by selecting a color component sub image, wherein color of thecolor component sub image selected and the color of the mark image areidentical.
 10. An image processing method comprising: displaying a markimage in a specific image of a stereo image; capturing the specificimage of the stereo image; and detecting the mark image by analyzing thespecific image that is captured.
 11. A computer-readable recordingmedium having stored thereon a computer program for executing an imageprocessing method, wherein the image processing method comprises:displaying a mark image in a specific image of a stereo image; capturingthe specific image of the stereo image; and detecting the mark image byanalyzing the specific image that is captured.
 12. The image processingsystem according to claim 3, wherein the image processing portion movesposition of the mark toward a gaze direction of the first camera portionso that the position of the mark in the specific image captured by thefirst camera portion remains at the center of the specific image. 13.The image processing system according to claim 3, wherein the imageprocessing portion adjusts size of the mark in proportion to distancebetween the camera portion and the mark so that the size of the mark inthe specific image captured by the first camera portion remains at aconstant value.